Agents, AI, and Azure AI Foundry: A Deep Dive into Enterprise AI Platforms

Session Type: STUDIO14 - Interview
Format: Online Only
Speakers: Yina Arenas (VP Azure AI Foundry, Microsoft), Marco Casalaina (VP Products & AI Futurist Core AI, Microsoft), Pablo Castro (CVP & Distinguished Engineer AI Platform, Microsoft), Seth Juarez (Principal Program Manager, Microsoft)
Link: [Microsoft Build 2025 Session STUDIO14]

Azure AI Foundry and Enterprise AI

Executive Summary

This intimate studio interview explores the cutting edge of enterprise AI through Azure AI Foundry and Azure AI Search. The conversation reveals how AI is evolving from simple RAG implementations to sophisticated agentic systems that can adapt, plan, and make decisions. The discussion covers practical applications including BMW’s real-world sensor data analysis system, the explosion of AI models, and how Microsoft is building the infrastructure to make AI accessible while keeping it safe and secure.

Key Topics Covered

?? 1. The Data Foundation: AI as Language Calculators

The Core Principle

Seth’s Analogy: “Models are language calculators - if you don’t put the right numbers in, the right numbers won’t come out.”

The Data Discipline Challenge:

  • AI amplifies existing data practices - poor data discipline becomes magnified problems
  • Quality data is non-negotiable - models will confidently provide wrong answers with bad data
  • Retrieval accuracy determines model effectiveness - finding the right information is crucial

Azure AI Search Role in Data Quality

Pablo Castro’s Perspective:

  • Job definition: “Find the right bit of information so the model knows what to do next”
  • Challenge: “Data comes out in all sorts of ways - people aren’t thinking about making it easy for indexing”
  • Solution approach: “Point us at your data - if you don’t want opinions, we got you”

?? 2. Advanced Document Processing and AI-Enhanced Indexing

Document Cracking Innovation

Automated Data Processing:

  • Multi-format support - PDFs, images, text, mixed media documents
  • Layout understanding - AI extracts structural information and context
  • Visual content analysis - pictures and diagrams become searchable content
  • Chunking optimization - intelligent document segmentation for better retrieval

AI-Powered Indexing Pipeline

Comprehensive Processing Stack:

  • Automatic vectorization - semantic embeddings without manual intervention
  • Layout preservation - document structure maintained for context
  • Multi-modal understanding - text, images, and structure processed together
  • Opinion flexibility - full automation or custom configuration options

?? 3. Evolution Beyond RAG: Agentic Retrieval

Traditional RAG Limitations

Historical Context:

  • Two years of RAG success - established patterns and applications
  • Learning accumulation - understanding what works and what doesn’t
  • New capabilities emerging - time for next-generation approaches

Agentic Retrieval Breakthrough

Pablo’s Innovation:

"We apply the same agentic methods to the search stack - 
agentic retrieval that can understand, reflect on what we got, 
see if we need more information, process and branch out queries"

Key Capabilities:

  • Reflective analysis - AI evaluates retrieval quality and completeness
  • Dynamic query expansion - automatic query reformulation and branching
  • Context-aware iteration - continuous improvement of search results
  • Multi-step reasoning - complex information gathering strategies

Advanced Ranking and Re-ranking

Deep Learning Integration:

  • Transformer-based re-ranking - modern neural models for relevance scoring
  • Million-to-five filtering - massive document sets narrowed to essential results
  • Semantic understanding - meaning-based rather than keyword-based ranking
  • Full semantic ranking stack - automated optimization for developers

?? 4. The Model Explosion: 10,000+ Models in Azure AI Foundry

Scale of Model Ecosystem

Yina’s Model Catalog Overview:

  • 10,000+ models available - comprehensive ecosystem across all domains
  • Historical growth - from 3 OpenAI models to massive diverse catalog
  • Specialized coverage - text, speech, image, video, industry-specific models
  • Domain expertise - healthcare, finance, retail, and specialized verticals

Model Selection and Discovery

Built-in Navigation Tools:

  • Catalog organization - multiple ways to slice and categorize models
  • Leaderboard comparisons - cost, throughput, safety, quality metrics
  • Scenario-based filtering - reasoning, text processing, image analysis
  • Use-case optimization - find the right model for specific applications

Model Router: Intelligent Automation

Automatic Model Selection:

"Model router is an overlay on deployed models - based on the prompt, 
it decides which model to use. Simple prompt ? Nano model (cheaper). 
Complex reasoning ? O3 model."

Dynamic Routing Benefits:

  • Cost optimization - automatic selection of most economical model
  • Performance matching - complexity-appropriate model assignment
  • Developer simplicity - no manual model selection required
  • Transparent operation - seamless routing without code changes

?? 5. Defining Agents: LLMs as Control Flow

Simple Agent Definition

Yina’s Core Concept:

"Agents is where you let a language model help you decide 
the control flow of the program."

Seth’s Developer Analogy:

  • New control structure - like “if”, “while”, “switch” statements
  • “Swift statement” - LLM-driven program flow control
  • Developer empowerment - higher-level abstraction for complex logic

The Adaptability Breakthrough

Traditional Automation Limitations:

  • Zero adaptability - hard-coded workflows break with changes
  • Maintenance nightmare - constant script updates for workflow modifications
  • Rigid interaction patterns - “representative, representative, representative”

AI-Powered Flexibility:

  • Dynamic adaptation - AI adjusts to changing requirements
  • Natural interaction - conversational interfaces replace rigid menus
  • Planning capability - AI can strategize multi-step approaches
  • Learning integration - systems improve through interaction

Function Calling and Tool Integration

API-Driven Agent Capabilities:

  • Universal tool access - anything describable as API becomes available
  • Azure AI Search integration - knowledge retrieval as agent capability
  • Action execution - real system interactions and modifications
  • Composable functionality - mix and match tools for complex workflows

?? 6. Emerging Protocols and Standards

The Standards Evolution Challenge

Early Technology Parallels:

  • Pre-HTTP analogy - standards are still evolving in agent space
  • Market consolidation - some protocols gaining more traction
  • Platform integration - supporting multiple emerging standards

Key Protocol Support

Agent-to-Agent Communication:

  • A2A Protocol - agent interaction and coordination standards
  • Multi-agent orchestration - complex workflow coordination

Tool Integration Standards:

  • Model Context Protocol (MCP) - standardized tool calling interface
  • OpenAPI integration - standard REST API connectivity
  • Custom protocol support - flexibility for proprietary systems

Azure AI Foundry Protocol Coverage

Comprehensive Standards Support:

  • MCP and A2A native support - leading protocol implementations
  • Assistants API compatibility - OpenAI standard integration
  • LangChain and CrewAI support - popular framework compatibility
  • “Bring your stuff” philosophy - existing toolchain integration

??? 7. Safety, Security, and Quality Assurance

Multi-Dimensional Safety Approach

Quality Evaluation Framework:

  • Traditional evaluators - relevance and accuracy assessment
  • Agent-specific evaluation - tool calling correctness validation
  • Intent understanding - system prompt adherence monitoring
  • Instruction following - behavioral compliance assessment

Security and Protection

Comprehensive Security Stack:

  • Prompt shields - protection against injection attacks
  • Continuous monitoring - real-time security assessment
  • Attack response - automated defense against adversarial inputs
  • Quality optimization - ongoing performance improvement

Development Environment Integration

Azure AI Foundry Safety Features:

  • Built-in evaluators - integrated quality assessment tools
  • Security umbrella - comprehensive protection across applications
  • Monitoring dashboards - real-time safety and performance metrics
  • Iterative improvement - continuous optimization workflows

?? 8. Agent Service: Production-Ready AI Microservices

General Availability Announcement

Production-Ready Agent Deployment:

  • Declarative agent definition - simple configuration-based creation
  • Cloud-native scaling - automatic resource management
  • Zero infrastructure management - focus on agent logic, not operations

Agent Configuration Components

Simple Agent Definition:

Agent Components:
??? Agent Identity: Name and personality definition
??? Instructions: Behavioral and operational guidelines  
??? Tool Integration: APIs, functions, and services
??? Data Sources: Azure AI Search, Fabric, SharePoint
??? Knowledge Access: Bing integration for world knowledge
??? Action Capabilities: Logic Apps, Azure Functions, OpenAPI

Integration Ecosystem

Comprehensive Connectivity:

  • Azure services - native integration with Microsoft ecosystem
  • Third-party APIs - OpenAPI standard support
  • MCP servers - standardized tool protocol support
  • Custom functions - Azure Functions for specialized logic
  • Data platforms - Fabric, SharePoint, and enterprise data sources

?? 9. Real-World Customer Implementation: BMW Case Study

The BMW Sensor Data Challenge

Scale and Complexity:

  • 5,000 sensors per vehicle - comprehensive data collection
  • Global fleet monitoring - worldwide data aggregation
  • Multi-dimensional data - engine, brake, ambient temperature, moisture
  • Azure cloud storage - centralized data lake architecture

The Access Problem

Traditional Data Silos:

  • “Special class of wizards” - only experts could query data
  • Cryptic sensor names - “Q underscore RSTR” meaningless to users
  • Kusto and SQL complexity - technical barriers to data access
  • Knowledge bottleneck - limited organizational data utilization

AI-Powered Solution

Semantic Model Development:

  • Six-month semantic modeling - comprehensive sensor definition project
  • Organization-wide collaboration - cross-team knowledge gathering
  • Sensor documentation - names, purposes, ranges, and relationships
  • Azure AI service integration - data agent implementation

Transformative Results

Universal Data Access:

Demo Query: "Show me all the hard-braking events in the last week in rainy weather"
Result: Successful cross-referenced analysis of sensor data, weather conditions, and vehicle events

Organizational Impact:

  • Democratized data access - anyone at BMW can query sensor data
  • Natural language interface - no SQL knowledge required
  • Real-time insights - immediate answers to complex questions
  • Scalable knowledge - AI agent handles infinite query complexity

?? 10. Platform Integration and API Management

Live Demo: Travel Agent API Integration

Rapid Development Workflow:

  • Yesterday’s requirement - flight reservation API integration needed
  • Azure API Management discovery - leveraging existing platform services
  • Mock API creation - rapid prototyping and development
  • OpenAPI protocol exposure - standard interface generation

Multi-Protocol API Exposure

Flexible Integration Options:

  • OpenAPI standard - immediate agent compatibility
  • MCP server creation - standardized protocol support
  • Multiple exposure methods - different protocols for different needs
  • Platform-native tools - Azure services working together seamlessly

The Integration Philosophy

Platform Cohesion:

  • Azure ecosystem synergy - services designed to work together
  • Developer productivity - rapid integration without complex setup
  • Standards compliance - industry protocols supported natively
  • Scalable architecture - enterprise-ready integration patterns

Technical Architecture and Innovation

Azure AI Search Evolution

Traditional ? AI-Enhanced Search:

  • Inverted indexes ? Vector embeddings + traditional search
  • Keyword matching ? Semantic understanding
  • Static results ? Dynamic re-ranking with deep learning
  • Manual configuration ? AI-powered automation

Agent Service Architecture

Microservice Approach:

  • Declarative definition - configuration over code
  • Cloud-native scaling - automatic resource allocation
  • Protocol abstraction - multiple standards supported transparently
  • Integration ecosystem - comprehensive tool and data connectivity

Safety and Quality Framework

Multi-Layer Protection:

  1. Input validation - prompt shields and injection protection
  2. Process monitoring - tool calling and instruction compliance
  3. Output evaluation - quality, relevance, and safety assessment
  4. Continuous improvement - feedback loops for optimization

Session Highlights

“Models are language calculators - if you don’t put the right numbers in, the right numbers won’t come out.” - Seth Juarez

“AI only amplifies the data estate that you have. If you are not disciplined with your data, AI is going to make you seem not disciplined.” - Seth Juarez

“Agents is where you let a language model help you decide the control flow of the program.” - Yina Arenas

“We have more than 10,000 models in the Azure AI Foundry catalog… models for all kinds of scenarios.” - Yina Arenas

“Show me all the hard-braking events in the last week in rainy weather” - and it can totally pull that off.” - Marco Casalaina (BMW demo)

Implementation Insights

Getting Started with Azure AI Foundry

1. Model Selection Strategy

**Model Router Approach:**

- Start with model router for automatic selection
- Let AI choose appropriate model based on prompt complexity
- Cost optimization through intelligent routing
- Performance matching without manual configuration

**Manual Selection Criteria:**

- Use catalog for specific domain requirements
- Leverage leaderboard for cost/performance optimization
- Filter by scenario type (reasoning, text, images)
- Consider industry-specific models for specialized use cases

2. Agent Development Process

**Agent Service Setup:**
1. Define agent personality and instructions
2. Configure tool integrations (APIs, Azure Functions)
3. Connect data sources (AI Search, Fabric, SharePoint)
4. Set up evaluation and monitoring
5. Deploy to production with automatic scaling

**Integration Patterns:**

- Use MCP for standardized tool protocols
- Leverage OpenAPI for REST service integration
- Connect Azure services natively
- Implement custom functions for specialized logic

3. Data Preparation for AI Success

**Azure AI Search Implementation:**
1. Point at data sources with minimal configuration
2. Enable automatic document cracking and processing
3. Use AI-powered indexing for multi-format content
4. Configure semantic ranking for optimal results

**Best Practices:**

- Invest in semantic modeling for domain-specific data
- Document sensor names, APIs, and data structures
- Create comprehensive metadata for AI understanding
- Enable agentic retrieval for complex information needs

Business Impact and ROI

BMW Transformation Results

  • Democratized analytics - from “wizard-only” to organization-wide access
  • Natural language querying - no SQL expertise required
  • Real-time insights - immediate answers to complex operational questions
  • Scalable knowledge - AI handles unlimited query complexity

Development Productivity Gains

  • Rapid prototyping - API creation and integration in hours, not days
  • Automatic optimization - model routing and cost optimization
  • Reduced maintenance - adaptive agents vs. brittle automation scripts
  • Platform integration - Azure services working seamlessly together

About the Speakers

Yina Arenas
VP Azure AI Foundry
Microsoft
Leads Microsoft’s strategy for AI model ecosystems, enterprise AI agents, and AI developer experience. Platform builder focused on enabling developers to integrate AI with trust and transparency.

Marco Casalaina
VP Products & AI Futurist, Core AI
Microsoft
Leads AI Futures team developing next-generation AI products. Previously led Azure OpenAI, Vision, Speech, and other Core AI teams. Former Einstein AI lead at Salesforce.

Pablo Castro
CVP & Distinguished Engineer, AI Platform
Microsoft
Corporate Vice President leading Azure AI Search team. Expert in information retrieval, machine learning, and distributed systems. Co-founder of Lagash Systems (acquired by Mercado Libre).

Seth Juarez
Principal Program Manager
Microsoft
Program Manager in Azure AI Core Product Group focusing on AI and Machine Learning. Expert in making complex AI concepts accessible to developers and businesses.

This studio session reveals how Microsoft is building the fundamental infrastructure for enterprise AI - from intelligent data retrieval through Azure AI Search to sophisticated agent orchestration via Azure AI Foundry. The conversation demonstrates that successful AI implementation requires not just powerful models, but comprehensive platforms that handle data quality, safety, security, and seamless integration with existing business systems.